A Free Energy Minimization Framework

This paper studies the task of inferring a binary vector s given noisy observations of the binary vector t = As modulo 2, where A is an M N binary matrix. This task arises in correlation attack on a class of stream ciphers and in the decoding of error correcting codes. The unknown binary vector is replaced by a real vector of probabilities that are optimized by variational free energy minimization. The derived algorithms converge in computational time of order between wA and N wA, where wA is the number of 1s in the matrix A, but convergence to the correct solution is not guaranteed. Applied to error correcting codes based on sparse matrices A, these algorithms give a system with empirical performance comparable to that of BCH and Reed-Muller codes. Applied to the inference of the state of a linear feedback shift register given the noisy output sequence, the algorithms ooer a principled version of Meier and Staaelbach's (1989) algorithm B, thereby resolving the open problem posed at the end of their paper. The algorithms presented here appear to give superior performance. (As + n) mod2 = r (1) where A is a binary matrix. Our task is to infer s given r and A, and given assumptions about the statistical properties of s and n. This problem arises, for example, in the decoding of a noisy signal transmitted using a linear code A. As a simple illustration, the (7,4) Hamming code takes N = 4 signal bits, s, and transmits them followed by three parity check bits.